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1, INTRODUCTION 

In general, a Constraint Satisfaction Problem (CSP) consists of a finite set of variables; each one has 
a finite domain of values and a set of constraints which imposes a variable domain restriction. And the goal 1s 
finding a complete assignment of variables which satisfies all constraints. Many problems from the real 
world can be reformulated as CSP. For example, qualitative and symbolic reasoning, diagnosis, scheduling, 
spatial and temporal planning, hardware design and verification, real-time systems and robot planning, etc. It 
is known that solving a CSP with a finite domain is an NP-complete problem, which requires a combination 
of heuristics [1] and combinatory search methods, in order to be solved in a reasonable time [2]. Mainly, 
approaches to solve CSPs can be classified into two categories: exact approaches and heuristic ones. As for 
exact approaches, most of them have the backtracking algorithm (BT) as a main algorithm for solving 
constraint satisfaction problems. The BT applies a Depth-First search in order to instantiate variables and a 
backtrack mechanism when dead-ends appear. Many works have been devoted to improve its forward and 
backward phases by introducing look-ahead and look-back schemes. Some other search algorithms for 
classical CSPs like Forward Checking, Partial Look-ahead, Full Look-ahead, and Really Full Look-ahead are 
introduced and their performances are studied widely in the literature [3-6]. For small problems exact 
method are better but for big instances of NP-Hard problem like CSP need high time cost, due to all methods 
have at last experiential complexity. In this case non exact approaches give an acceptable solution in 
reasonable times. As far as heuristic approaches are concerned, we find that, a very different approach has 
investigated discrete Hopfield Neural Network (HNN) to solve CSP [7]. In this neural network approach, the 
CSP constraints are associated with the network topology, biases, and connection strengths. Recently, many 
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approaches introduced Continuous Hopfield network to solve CSPs [8-10] or problems which can be 
formulated as CSP [11]. As we can see in [10], the authors propose mapping CSP to a quadratic problem and 
elaborate an appropriate parameters setting of network energy, then they solve it by finding the network 
equilibrium point [8]. Based on the same convergence procedure, Calabuig [12] adds dynamically linear 
constraints to confine the network to the feasible subspace of solutions; this improvement ensures the final 
solution validity. Practically, there are two important problems with approaches based on conventional neural 
network architectures. The first problem is that HNN partially mitigates the problem of getting stuck in local 
optimum. The second one is due to Hopfield network dynamic which continuously explores the search space 
and will not always stabilize at border O or 1. If the same case appears, we get low solution quality or an 
incomplete assignment of variables problem. In order to tackle these problems, we propose to improve the 
CHN solution by the Min-Conflict heuristic(MNC) [13]. The MNC works as follows: it selects the value 
from each variable domain for which the total error in the next configuration will be minimal. Precisely, 
MNC is a repair-based stochastic approach, which starts with a complete but inconsistent assignment and 
then repairs variables implicated in constraint violations repetitively until a consistent assignment is 
achieved, this approach can solve large-scale problems in a practical time. 

This paper is organized as follows: In section 1, we present a modelization of the binary CSP as 0-1 
quadratic program, and generalizes energy function associated with the CSPs. Section 2 is devoted to giving 
implementation details of the proposed approach. In the last section, we show the complexity analysis and the 
results of the numerical experiments against other approaches like the Genetic Algorithm [14-16]. 


2. BINARY CSPS SOLVED BY CHN 
2.1. Quadratic model of Constraint satisfaction problem 
A large number of real problems such as artificial intelligence, scheduling, assignment problem can 
be formulated as a Constraint Satisfaction Problem. Solving a CSP requires to finding an assignment of all 
variables problem under constraints restriction. The CSP can be formulated as three sets: 
1) Set of N variables X={X;;1<1i<N }. 
2) Set of N variables domains: D ={ D;; 1< 1<d,;} where each Di contains set of d; range values for 
Xi. 
3) Set of M constraints: C ={ Cj; 1< 1i1<M}. 
Each constraint C; associates an ordered variables subset which is called the scope of C;. The arity of 
a constraint is the number of involved variables. We can easily reformulate CSP as a Quadratic Problem 
(QP), by introducing a binary variable x;, for each CSP variable x;, where k varies over the range of x;, given 
as follows: 


(1) 


1, if variabla takesvaluek 
Xik = ; 
0, otherwise 


For each binary constraint Cij , between the variables yi and yj , we associate a state function 
defined as: 


ce 
S;j(%) = > 3'x,4 pO. (2) 


f=) s=) 


Where ~~ (Xx fF SLL. NIK ed; } a vector of QP solution and the quadratic terms Gin ” defined as: 


(3) 


QO otherwise 


From all the equations defined in (2), which correspond to problem constraints, we deduce the 
objective function of its equivalent QP: 


ndawn d . 
fQ)= PP Geis 4) 


i=1 r=] j=l s=1 
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Furthermore, some strict linear constraints equations must be satisfied by the solution: ae pir =I; 


for 1=1..N which can be written also as Ax=b (A isa NxM matrix and b isa ™ dimension vector 
fully initialized to ). So, the model is given as follows: 


Min f(x) =x' Ox 
p with 
(QP) ee 
xe{0,1} (5) 


Systematically, to solve the last Quadratic Optimization Problem with Hopfield model, we need to 
build an energy function such as the feasible solutions of the problem corresponding to the minimal of CHN 
energy function. 


2.2. Hopfield neural network 

Hopfield neural network was introduced by Hopfield and Tank [17] [18] [19]. At the beginning, it 
was developed to solve combinatorial optimization problems. Then it was extended extensively to other areas 
of application, such as recognition and optimization. Hopfield neural network is classified as an efficient 
local search, which gives an acceptable solution to hard optimization problems at a reasonable time. 
Basically, this neural network model is a fully connected neural network and its dynamic stats are governed 
by the following differential equation: 


CA ee (6) 
dt c 


Where: 
e x: vector of neurons input 


e y: vector of output 


e T : matrix of weight between each neurones pairs 
e i? : The biases which describe the neurons self-internal noises. 


Hopfield proved that if T is symmetric then the network energies is a Lyapunov function: 


1 “1 /* 
E(2) = —52'Tr - (i°)'r + S- = / g '(v)du 
~ im1 * 40 
(7) 


For each neurons, the input is governed by an activation function x = g(y) which varies between 0 
and 1. This function is given by: 


g(y) = 5 (1+tanh2)) (8) 
Ug 


We define the energy function of CHN as to be associate with the cost of the problem to be 
minimized; this is down by taken T too large: 


E(x) =x'Tx+(i?)' x (9) 


In this paper, we use the ability of CHN to resolve Quadratic model optimization. So we choose a 
CHN energy function similar to [10], which is adapted to solve the quadratic formulation of the binary CSPs: 
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NN G d . N 4 N 4 
E(x) = 25> yim, Ojjrs + BY Sx, + YD xp I—x;,) (10) 
1 


i=] j=lr=ls= i=l r=1 i=l r=] 


The first term penalizes two values taken by two linked variables which caused a violation, the 
second term restricts each variable to take one value, the third term aggregates all linear constraints (Ax = b), 
and the last term satisfies the propriety of integrity which ensures neurons to stabilize at O or | state. By 
corresponding (9) and (10), we deduce the link weight and biases of the network. 


Ls = —a( 1-65) dirs —0;, 9+ 20,057 (11) 
-b _ 
ir = —B-y 
| ifjej 
with 0;, = y ] is the Kronecker delta. 
J 10 ifiFj 


To obtain an equilibrium point of the CHN, the parameters setting procedure is used [10] [8]. The 
Principe of this procedure is to move the network in the direction of —. This process speeds up the neural 


network convergence significantly. Furthermore, to assure the stability, we use the hyperplane analysis 
; OE(x ae 
method which calculates the energie function evolution. Let Ej, = ~~ , the following conditions must be 
ir 





imposed to ensure the convergence. 
¢ > 0 to have a minimization problem 


e The first one concerns avoiding the case when two values are assigned to the same variable, Ji, 7 € 
D(X) with i # j and their corresponding neurons X4j=Xx,;=1, so E(x) must verify: 


E. (x)22o+B—-y2e (12) 


e The second ensures that all variable must be assigned and escape the case where x;,=0 r € {1,...,d;} 
so E(x) must verify also: 


E. (x)Sad+Bp+ys-€ (13) 


With: 


N d; 
= Mar} So >. dirjs /ie {1,...,N} et r € {1,...,di} 
j=1 s=1 


Empirically, the best value of € is 10° parameter setting by solving [10] 


~g=0, a>0 

—p+2v 20 

20+ B-V=E 
ad+B+y=-é 


(14) 


The CHN parameters setting and the starting point used in this paper are similar to [10]. 
Furthermore, a procedure which allows each neuron of the same variable to take 0, if one converges to the 
boarder 1, is added. This procedure is called Moderator (Figure 1). 
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Function Moderator( x; : Current neuron } : 


If (rx, = 1) then 
For each 7 € Dom(V;) \k do 


| x5 — 0 
end For 
end If 
End 





Figure 1. Moderator 


This approach, which only uses Continuous Hopfield neural Network (CHN), was extensively 
studied over different instances of Constraints satisfaction problem [8], but simulation proved that this 
approach has many disadvantages. For example, sometimes the network in continuous dynamic does not 
gives a valid solution, and it is attracted by the nearest local minimal to the starting point. This attracting 
effect remains a result of its deterministic input-output interaction of the units in the network. Consequently, 
the network is not able to escape from a local solution closed to the starting point. Also, for Hopfield neural 
network with a continuous dynamics case, each unit output can take any value between 0 and 1. So, the 
network can be stranded at a local minimum which contains some units that still take real values. In this case, 
we obtain an invalid solution for CSP. To overcome this neural network weakness, the main idea of this work 
is to repair the solution given by the CHN and improve it by a known Min-conflict algorithm. 


3. CHN AND MIN CONFLICT HEURISTIC TO SOLVE CSPS 

There are many methods which combine two or more no exacts approaches to solve a given 
optimization problem [11, 20—23]. In the same direction we introduce a hybrid approach based CHN and 
MNC. The MNC algorithm [13] is a very simple and fast local repairing method to resolve CSPs, which aims 
at assigning all the variables randomly. Next, it iteratively selects one variable from the set of the variables 
with conflicts which violates one or more constraints of the CSP. Then, it assigns a value to the selected 
variable, so that it can minimize the number of conflicts. MNC has demonstrated to be able to solve the 
queens problem in minutes [24]. MNC is widely used to construct hybrid algorithms with other optimizations 
[25-28]. In this way, the basic idea of our proposed approach is to use MNC to improve the solution reached 
by CHN. This will be done in tw o steps (see Figure 2). First, MNC visits all assigned variables; for each one, 
we apply Min-Conflict directly to the neural network structure, then, it returns the best assignment for the 
current variable (see Figure 3), the decision will be taken by the sum of all activated neurons weight. Second, 
we propagate this assignment to other set variables not yet assigned iteratively by applying the MNC 
heuristic to guarantee as much consistency as possible. The diagram of our proposed algorithm is described 
by (Figure 4). 


Function CHNMNC ( CSP: Problem ): 


Ve = CHN(CSPs) 

For (each cluster 7; € V,) do 
se! (x j,a@) {a is the current affected value to 2; } 
a = Min-Conf(2;,Vqa) { new value assigned to 2; } 


Va = Val (23,2) 


end For 
{propagate current sub assignment to other 
variables which not assigned } 
For (each cluster 7; ¢ Va) do 

a’ = Min — Conf (2x;,Va) 

_ r P 2 = 

Va = Va _}(2z;,4 ) 
end For 
return V, 


End 





Figure 2. Main function which improve solution by Min-conflict algorithm 
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Function Min-Conf ( 2; : Current variable , V, : Assigned 
variables set ): 


let x}: the current position which is assigned to the variable 2; 


For (each value vz; € Dom(2;)) do 
‘ »)} — me TC, 
w(k )= Page Vis Tikjs 
~*~ 1 > yaatic gc0ciT - rari 1 >. 
{ x}, = 1 is the position assigned to variable x; } 


end For 
let Winrar set of maximum output wait 


If (x7; € Pmazx ) then 

| return r 
else 

| return random value from Pmax 
end If 


End 





Figure 3. Selected the most coherent neuron of current cluster with other variables clusters already affected 
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Figure 4. Diagram of the proposed algorithm 
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Example 1: Meeting Scheduling Problem 
The Meeting Scheduling problem (MSP) is a decision-making process affecting several people, in 
which it is necessary to decide when and where several meetings could be scheduled. To show how this 
algorithm works, we consider the following MSP example: 
¢ Persons={0,1,2,3} 
¢ Meetings: 
¢ m, attend person group: {0,1} 


1 
> mM, attend person group: {1,2} 
° m3 attend person group: {0,1,2} 
° my attend person group: {2,3} 


¢ Location L={ pl), pl, pl3, pla} 
¢ Distances : 





¢ Time Slot={ t1, tr, t3, ta ts} 
¢ Duration of each meeting I time slot 


This problem can be reformulated as the following CSP: 
¢ Variables: V={ m), m2, m3, m4} list of meeting 
¢ Domains D=={ t), to, t3, t4 ts} where 
t;=tg={0,1,2,3,4} and to=t3={0,4} 
¢ Constraints C= {C;, 1 <i<j <4} 


et {0, 1, a 3,4} and tg —t3 = {0,4} 


We supposed that CHN after running gives an incomplete solution S =v {m yal i {m5=4 ifm 379} ) 


First pass : For each variable in v 


e is 
© w(0)=w(1)=w(3)=w(4)=-a 
© w(2)=0 
so Min-Conf(m,, V,,) return 2 
e Lo, 
© w(O0)= -a 
© w(4)=0 
So Min-Conf(m2,V,) return 4 
e its 
¢ w(0)=0 
© w(4)=- a 


so Min-Conf(m3, V,,) return O 


The second phase will take m y, which is not yet assigned : 


e | 
© w(0)=w(l)=w(3)=w(4)= -a 
° w(2)=0 
so Min-Conf(m4, V,,) return 2 


The final solution is S={{m ,=2},{m 2=4},{m3=0}, {mg=2}, }. 
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4. NUMERICAL RESULTS 
4.1. Generated randomly instances 

To evaluate the performance of proposed resolving methods, we run some preliminary experiments 
on the randomly generated problems and we compare the solution quality by the number of violations. 
Simulation has been done with the following machine characteristics: IS core(TM) 2,35GHz processor and 
limitation of memory to 2GO. For CHN parameters, the best values were founded empirically: 


d=ad+2e y= B=e-37 


bo |- 


To generate random problems, we use a random generator based extended model B as it is described 
in [29-31]. This extended model which is called Model RB is able to generate a hard satisfiable instance. To 
summarize this generation method, a random CSP 1s defined by the following five input parameters: 


¢ k: denotes the arity of each constraint, fixed at k=2, so all constraints are binary 
¢ mn: denotes the number of variables, 

¢ : determines the domain size d=n° of each variable, 

e r>0: determines the number m=rnIn(n) of constraints, 


¢ 1>p>0: determines the number t=p d* of disallowed tuples of each relation. 


In the following, a class of CSPs will be denoted by a tuple of the form <n,d,m,k,p>. For simulation, 
we generate 50 instances for each p value for k=2, a =0.8, r=3 and n in {20, 30, 40}, and we calculate the 
average of violated constraints of each instance in the same class of p value. For the model parameter used, 
we get <20,11,180,p>, <30,15,306,p> and <40,19,443,p>. The corresponding load curve is given in Figures 
5, 6 and 7 respectively. Comparison is made between the solution obtained by the proposed algorithm and the 
solution given by the original approach [10]. As we can see Min-Conflict improves the quality of the solution 
considerably around mean 50%, and the success of network is up to 100%, but for CHN approach, we find 
the empirically mean value 72% over all runs. 





60 
seeks * CHNMANC 
50 a. 7 
F * 22 = CHN 
= a 
> 40 * * 
al a : 
= » 
- a = 7 
- 30 - : . 
= + 
3 a 
F 20 ak 
Ee j x 
— | = . 
= 10 : ; . . 
. *., = 
+ .* + *e, . 
o ix --—- 
10 20 30 40 50 60 70 B0 90 
Tightness (p) 


Figure 5. number of violation over classe N=20 
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Figure 6. number of violation over classe N=30 
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Figure 7. number of violation over classe N=40 


4.2 Typical instances 

For showing the practical interest of our approach, we also study its performance over problems of 
different natures (random, academic and real-world problems) [32]. The goal is to evaluate its performance 
with other evolutionary algorithms [33, 34]. Thus, we compare its efficiency with the Genetic Algorithm 
(GA) [35] and the Particular Swarm Optimization (PSO) [36, 37]. In practice, rather than authors settings we 
have empirically searching the Genetic Algorithm and PSO goods ones, adopted to CSP problem. So, for GA 
the population was 200 individuals, mutation rate equals to 5% and crossing rate equals to 72%, as for PSO 
[36, 37] we chose ~, = @~2 = 1 and population size fixed at 100. We also run each one 200 times. 

The CHN parameters setting and the starting point used in this paper are similar to [10], we have 
used the Variable Updating Step (VUS) technique proposed by Talavan and Yafiez in [8]. Table 1 shows the 
comparison between our approach CHN-MNC and original CHN. The description of table columns is the 
following: 

e V: the number of variables. 

¢ C: the number of constraints. 

¢ Ratio mean: the average of the optimal value in a 200 run. 
¢ Time: the average of the time of all runs. 
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Table 1. Performance of our proposed methods CHN-MNC and CHN over typical instances problems 


Name of V C CHN-MNC GA PSO 
Instance Mean CPU(s) Mean CPU(s) Mean CPU(s) 

queens-10 10,00 45,00 1,00 0,01 2,00 0,02 13,00 0,04 
queens-20 20,00 190,00 2,00 0,03 4,00 0,06 39,00 0,11 
queens-5-5-5 25,00 160,00 0,00 0,03 0,00 0,06 19,00 0,10 
frb30-15-5-mgd 30,00 210,00 10,00 1,07 14,00 2,46 40,00 4,08 
geom-3(0a-5 30,00 81,00 1,00 0,06 2,00 0,13 7,00 0,21 
geom-3(0a-6 30,00 81,00 0,00 0,05 0,00 0,12 4,00 0,20 
queens-30 30,00 435,00 4,00 1,94 6,00 4,45 83,00 7,39 
frb40-19-3-mgd 40,00 308,00 14,00 0,93 21,00 2,14 53,00 3,56 
geom-40-2 40,00 78,00 23,00 0,01 24,00 0,03 28,00 0,04 
geom-40-6 40,00 78,00 0,00 0,09 0,00 0,21 4,00 0,36 
myciel-5g-3 47,00 236,00 10,00 0,22 12,00 0,50 47,00 0,83 
myciel-5g-4 47,00 236,00 5,00 0,41 8,00 0,95 29,00 1,58 
myciel-5g-5 47,00 236,00 1,00 0,47 3,00 1,09 12,00 1,81 
myciel-5g-6 47,00 236,00 0,00 0,13 1,00 0,31 12,00 0,51 
driverlogw-Olc 71,00 217,00 0,00 0,08 0,00 0,17 3,00 0,29 
composed* 105,00 620,00 13,00 2,93 14,00 6,75 56,00 11,20 
dsjc-125-1-4 125,00 736,00 50,00 0,30 64,00 0,69 102,00 1,15 
dsjc-125-1-5 125,00 736,00 19,00 0,80 29,00 1,83 85,00 3,04 
Qw h-15-106-1 225,00 2324,00 20,00 1,60 23,00 3,68 66,00 6,11 
qwh-15-106-4 225,00 2324,00 18,00 4,06 22,00 9,35 59,00 15,52 
qwh-15-106-6 225,00 2324,00 22,00 2,31 23,00 5,31 68,00 8,82 
driverlogw-04c 272,00 3876,00 3,00 8,32 5,00 19,13 11,00 31,75 
driverlogw-02c 301,00 4055,00 3,00 9,17 5,00 21,09 9,00 35,01 
qwh-20-166-0 400,00 5092,00 30,00 16,79 32,00 38,63 93,00 64,12 
qwh-20-166-3 400,00 5092,00 29,00 2,88 31,00 6,63 89,00 11,00 
qwh-20-166-6 400,00 5092,00 25,00 8,19 29,00 18,84 86,00 oy 
le-450-5a-3 450,00 5714,00  1173,00 333,69 1261,00 767,49 1566,00 1274,03 
le-450-5a-4 450,00 5714,00 712,00 3,89 745,00 8,95 1066,00 14,86 
le-450-5a-5t 450,00 5714,00 441,00 2,29 470,00 5,26 874,00 8,72 


NB:composed* is the instance composed-25-10-20-5 


Table 1 we learn that GA and CHN-MNC are close and both better than PSO, but regarding the 
mean time taken by all compared algorithms CHN-MNC is the short one. GA [35] have the same principle 
with our approach while they use GA and Minimization of conflicts, but they improve the best individual. 
It’s not sure that improving a best solution will give the good one, it can be near the worst one in the 
population. For this reason, the means value of the multiple runs of our approach was the best because it 
improves each founded solution. 


5. CONCLUSION 

In this paper, we have proposed a new approach for solving binary constraint satisfaction problems. 
Our hybrid algorithm gives a good solution quality rather than using CHN alone, this improvement is done 
by adding a no importuned time computation. Furthermore the rate of network success to give a valid 
solution is up to 100% by repairing. Also, the results of the numerical example show that our approach is 
competitive with GA and PSO. The basic Hopfield neural network architecture described above includes 
only binary constraints, but some classes of problems are natively n-ary. The extension of this approach to 
solve these problems is possible if we can translate any n-ary problem to an equivalent binary one. Other 
studies are in progress to apply this approach to many real-world problems such as aircraft conflict, 
timetabling and Meeting scheduling. 
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